Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization

Stochastic compositional optimization generalizes classic (non-compositional) stochastic to the minimization of compositions functions. Each composition may introduce an additional expectation. The series expectations be nested. is gaining popularity in applications such as reinforcement learning and meta learning. This paper presents a new Stochastically Corrected Compositional gradient method (SCSC). SCSC runs single-time scale with single loop, uses fixed batch size, guarantees converge at same rate descent (SGD) for non-compositional optimization. achieved by making careful improvement popular method. It easy apply SGD-improvement techniques accelerate SCSC. helps achieve state-of-the-art performance In particular, we Adam SCSC, exhibited convergence matches that original on We test using model-agnostic meta-learning tasks.